Imagers employ either a two-dimensional (2D) multichannel detector array or a single element detector. Imagers using a 2D detector array measure the intensity distribution of all spatial resolution elements simultaneously during the entire period of data acquisition. Imagers using a single detector require that the individual spatial resolution elements be measured consecutively via a raster scan so that each one is observed for a small fraction of the period of data acquisition. Prior art imagers using a plurality of detectors at the image plane can exhibit serious signal-to-noise ratio problems. Prior art imagers using a single element detector can exhibit more serious signal-to-noise ratio problems. Signal-to-noise ratio problems limit the utility of imagers applied to chemical imaging applications where subtle differences between a sample's constituents become important.
Spectrometers are commonly used to analyze the chemical composition of samples by determining the absorption or attenuation of certain wavelengths of electromagnetic radiation by the sample or samples. Because it is typically necessary to analyze the absorption characteristics of more than one wavelength of radiation to identify a compound, and because each wavelength must be separately detected to distinguish the wavelengths, prior art spectrometers utilize a plurality of detectors, have a moving grating, or use a set of filter elements. However, the use of a plurality of detectors or the use of a macro moving grating has signal-to-noise limitations. The signal-to-noise ratio largely dictates the ability of the spectrometer to analyze with accuracy all of the constituents of a sample, especially when some of the constituents of the sample account for an extremely small proportion of the sample. There is, therefore, a need for imagers and spectrometers with improved signal-to-noise ratios.
Prior art variable band pass filter spectrometers, variable band reject filter spectrometers, variable multiple band pass filter spectrometers or variable multiple band reject filter spectrometers typically employ a multitude of filters that require macro moving parts or other physical manipulation in order to switch between individual filter elements or sets of filter elements for each measurement. Each filter element employed can be very expensive, difficult to manufacture and all are permanently set at the time of manufacture in the wavelengths (bands) of radiation that they pass or reject. Physical human handling of the filter elements can damage them and it is time consuming to change filter elements. There is, therefore, a need for variable band pass filter spectrometers, variable band reject filter spectrometers, variable multiple band pass filter spectrometers or variable multiple band reject filter spectrometers without a requirement for discrete (individual) filter elements that have permanently set band pass or band reject properties. There is also a need for variable band pass filter spectrometers, variable band reject filter spectrometers, variable multiple band pass filter spectrometers or variable multiple band reject filter spectrometers to be able to change the filters corresponding to the bands of radiation that are passed or rejected rapidly, without macro moving parts and without human interaction.
In several practical applications it is required that an object be irradiated with radiation having particularly shaped spectrum. In the simplest case when only a few spectrum lines (or bands) are necessary, one can use a combination of corresponding sources, each centered near a required spectrum band. Clearly, however, this approach does not work in a more general case, and therefore it is desirable to have a controllable radiation source capable of providing arbitrary spectrum shapes and intensities. Several types of prior art devices are known that are capable of providing controllable radiation. Earlier prior art devices primarily relied upon various “masking” techniques, such as electronically alterable masks interposed in the optical pathway between a light source and a detector. More recent prior art devices use a combination of two or more light-emitting diodes (LEDs) as radiation sources. In such cases, an array of LEDs or light-emitting lasers is configured for activation using a particular encoding pattern, and can be used as a controllable light source. A disadvantage of these systems is that they rely on an array of different LED elements (or lasers), each operating in a different, relatively narrow spectrum band. In addition, there are technological problems associated with having an array of discrete radiation elements with different characteristics. Accordingly, there is a need for a controllable radiation source, where virtually arbitrary spectrum shape and characteristics can be designed, and where disadvantages associated with the prior art are obviated. Further, it is desirable not only to shape the spectrum of the radiation source, but also encode its components differently, which feature can be used to readily perform several signal processing functions useful in a number of practical applications. The phrase “a spectrum shape” in this disclosure refers not to a mathematical abstraction but rather to configurable spectrum shapes having range(s) and resolution necessarily limited by practical considerations.
In addition to the signal-to-noise issues discussed above, one can consider the tradeoff between signal-to-noise and, for example, one or more of the following resources: system cost, time to measure a scene, and inter-pixel calibration. Thus, in certain prior art systems, a single sensor system may cost less to produce, but will take longer to fully measure an object under study. In prior art multi-sensor systems, one often encounters a problem in which the different sensor elements have different response characteristics, and it is necessary to add components to the system to calibrate for this. It is desirable to have a system with which one gains the lower-cost, better signal-to-noise, and automatic inter-pixel calibration advantages of a single-sensor system while not suffering all of the time loss usually associated with using single sensors.
The conventional spectral imaging systems can be generally categorized into three types: so-called pushbroom imagers, filter scanned imagers, and modulated focal plane array systems. FIG. 60 depicts a typical pushbroom spectral imaging system. In pushbroom imagers, a scene 8000 is imaged onto the entrance aperture of an imaging spectrograph containing a slit mask 8010. One column of spatial resolution elements defined by rows of detectors at the focal plan of an imaging spectrograph are spectrally imaged through the slit for each frame of data captured by the focal plane. The light from the slit is spectrally dispersed 8020 along a direction perpendicular to the direction of the slit, generating a rectangular image 8030 which varies spatially along an axis parallel to the slit, and spectrally along the orthogonal axis. A focal plane array is typically used to capture the rectangular image. To image a scene, the slit is translated such that it moves incrementally across the scene, with the pushbroom system collecting one image for each position of the slit. In this way, the pushbroom system collects a so-called hyper-spectral image cube of the scene. Note that it is standard in the art to refer to such a dataset as an image cube or a data cube or a datacube. This use of “cube” is imprecise, in that the underlying dataset may be rectangular (i.e. not necessarily of the same size in all 3 dimensions). The scanning of the slit across the scene may be accomplished by moving the imaging system, moving the scene or an object of interest, or optical scanning (e.g. with a moving (macro) mirror).
However, the slit must be very narrow in the pushbroom system to achieve the desired spectral resolution. At any given time, a narrow slit only accepts a very small fraction of the light from the entire scene, thus making the hyper-spectral imaging systems much less sensitive than the conventional imaging systems. That is, the pushbroom system will either acquire data of low signal to noise ratio or take longer to acquire the data. In the latter case, the pushbroom system is limited to imaging scenes that do not change over time. For changing scenes, the pushbroom system will suffer from artifacts.
Additionally, the scanning must be mechanically precise with the pushbroom system. Any scanning errors due to mechanical imprecision in the pushbroom system will result in a distorted data set. Further, the need to move the camera or the object of interest in certain pushbroom system is undesirable for a variety of reasons, including but not limited to the corresponding costs and complexities of associated additional system components, and the limitation that the system may only be used in contexts where it is possible to move the object or the camera in a controlled manner.
In filter scanned imagers, an ordinary broadband camera images a scene of interest, but a tunable filter is inserted somewhere in the optical path between the scene and the camera. The filter may be a liquid crystal tunable filter such as a CRI VariSpec LCTF (see. http://www.cri-inc.com/files/VariSpec_Brochure.pdf), or any similar device that transmits a narrowband of wavelengths at any given time, with the center wavelength of the band tunable in time. FIG. 31 depicts a typical liquid crystal tunable filter scanned spectral imaging system 3000. Light from the object 3010 enters the collimating lens system 3020 where it encounters the liquid crystal tunable filter system 3030 and then propagates onto the focusing lens system 3040 where the object is focused onto the focal plane 3050.
However, the bandpass of the filter in the filter scanned imager must be very narrow to achieve the desired spectral resolution. Like the slit in the pushbroom systems, at any given time, a narrow bandpass only accepts a very small fraction of the light from the entire scene, thus making the hyper-spectral imaging systems much less sensitive than ordinary imaging systems. That is the filter scanning system will either acquire data of low signal to noise ratio (SNR) or take longer to acquire the data. In the latter case, the filter scanning system is only limited to imaging scenes that do not change over time. For changing scenes, the filter scanning system will suffer from artifacts.
Multiplexed focal plane array spectral imaging systems, such as Fourier transform interferometric imaging systems, typically employ an imaging optical system, the output of which is passed through an interferometric assembly, and then imaged onto a focal plane array. As the interferometer is scanned, a multiplexed spectral image is acquired. FIG. 32 depicts a conventional scanning multiplexed focal plane array spectral imaging system, such as a Fourier transform focal plane array spectral imaging system 4000. The object source 4010 is collected by image grade collimating optics 4020 where it is collimated onto a beam splitter 4030 that splits the energy 50/50 to stationary mirror 4040 and to a moving mirror 4050. This is then recombined at the beamsplitter 4030 and propagates onto the focusing optics 4060 and is re-imaged onto the focal plane 4070.
However, in Fourier transform interferometric imaging and other similar systems, distortions and extreme system sensitivities can result from passing the light of an imaging system through an interferometer. This leads to the distorted data, as well as system complexity and extreme sensitivity of motion.
Each of the foregoing prior art systems scans through a full hyper-spectral datacube. However, the output of such measurement from such prior art system is generally an input to an algorithm that processes each pixel, and produces an answer consisting one or a few numbers per pixel. For example, these could be the output of a set of inner products, as is standard in the art of chemometrics. Since hyper-spectral datacubes contain a large amount of data and the answer consists of a smaller amount of data, it is desirable to find a method for directly measuring such answer. In other words, it is desirable to find a method for enabling adaptive measurements of spectral image parameters.
Conventional Hadamard transform spectroscopy (HTS), Hadamard transform imager (HTI), and Hadamard transform spectral imager (HTSI) overcome only some of the limitations and problems described herein. Hadamard optical systems utilize spatially encoded apertures that can be employed either at the entrance aperture of an optical system, the exit aperture or both. They have the common attribute that they encode the available aperture spatially where the spatial resolution elements that make up the encodement dictate the spectral, spatio-spectral or spatial resolution elements that propagate through the optical system including diffractive optical elements and on to the sensor or exit aperture. These masks have some spatial extent that places special requirements on the optics of the system. As the encodement mask grows either by longer length encodements with fixed sub-apertures or as the sub-aperture dimension grows for a fixed encodement length, the spatial resolution elements making up the sub-apertures in the encodement mask depart from the optical axis. When the resolution elements depart from the optical axis or paraxial condition it is desirable to employ optics that can image the off axis resolution elements without inducing excessive aberrations that degrade the performance or cripple the advantages gained by Hadamard transform (HT) multiplexing.
Typically the optical path for conventional monochromators begins with a source that is focused onto an aperture plane that has a large aspect ratio aperture known as a slit. This slit is often very small in extent in the dispersion plane compared to the other extent in the spatial plane. However, it is not required that this aspect ratio be large. If the aspect ratio is close to 1 then simple spherical optics can be employed that perform well as long as the departure from the optical axis is kept to a minimum. However, most monochromators have a large aspect ratio in order to increase the opportunity to maximize throughput, and detectors must be able to “see” the large extent of the slit aperture. The light entering the slit aperture is then dispersed and focused onto an exit slit aperture. Monochromators generally perform well on the optical axis and do not typically employ optics that can manage rays that depart from the optical axis in the plane of dispersion as required by HT multiplexing instruments. The optical system generally utilizes optical performance attributes normally found only in imaging and spectral imaging systems to employ encoding techniques. This requirement is driven by the extent of the encoding mask. The extent of the encoding mask is governed by the diffraction limit of the wavelengths within the bandpass, the encodement length N and the attributes of the optical system.
In a conventional dispersive spectrometer the radiation from a source is collected and separated into it's individual spectral resolution elements by a spectral separator such as a diffraction grating or prism and then is collected and focused for spatial presentation on a focal plane. The dispersive spectrometer uses a single exit slit to select one spectral resolution element of N spectral resolution elements for measurement by the detector. The Hadamard transform spectrometer (HTS) uses an array of slits (i.e. a mask) at the focal plane to select one more than half, (N+1)/2, of the spectral resolution elements at the focal plane for measurement by the detection system. The optical challenge to effect an HT multiplexing spectrometer is to collect all of the spatially distributed individual band pass images of the entrance slit and transfer them to as small detector as possible. It is desirable to keep the area of the detector at a minimum as the noise of many detectors increases with the square of the area. If the optics are able to illuminate a single detector element with all of the available light impinging upon the focal plane containing the spatially distributed images of the slit for each of the N band pass resolution elements, a multitude of spectral resolution elements can be measured simultaneously using a single detector element. This arrangement results in a multiplexing spectrometer. The recovery of N spectral resolution elements requires measuring the detector response for N different encodements of (N+1)/2 open mask elements. The raw data is recorded as the detector response versus encodement number and is called an encodegram. Hadamard transformation of the encodegram yields the spectrum.
The Hadamard transform instruments developed in the 1960s and 1970s employed moving masks. Significant problems such as misalignment and jamming associated with a moving mask led to a reputation of poor reliability and contributed to a dormant period in the development of Hadamard transform spectrometer (HTS) and Hadamard transform imager (HTI). Interest was rekindled in the 1980s using stationary Hadamard encoding mask based on liquid crystal (LC) technology. The first generation 1D stationary Hadamard encoding mask was a cholesteric LC with N=127 mask elements and used polarization as its operating phenomenon. Two parallel polarizers and rotation or lack of rotation of the polarized radiation generated the opaque and transparent states, respectively. The second generation 1D stationary Hadamard encoding mask was fabricated using a polymer dispersed liquid crystal (PDLC) material with N=255 mask elements and used light scattering as its operating phenomenon. The PDLC contained LC droplets dispersed in a polymer matrix whose index of refraction matched the index of refraction in one direction in the birefringent LC droplet. Alignment of the LC droplets optical axis under an applied voltage removed discontinuities in index of refraction at the polymer matrix/LC interface to generate a good transparent state while random orientation of LC droplets in the polymer matrix generated the opaque state from light scattering by the discontinuities in index of refraction at the polymer matrix/LC droplet interface. A 2D stationary Hadamard encoding mask was also based on LC technology. A fero-electric liquid crystal (FLC) positioned between a pair of polarizers with perpendicular orientation operated as an electro-optic half-wave plate when a + value of applied voltage rotated the plane of polarization by 90 degrees to produce the transparent state and a − value of applied voltage left the plane of polarization unaltered to produce the opaque state.
Development based on stationary Hadamard encoding masks continued in the 1990s and a 2D moving Hadamard encoding mask was also fabricated and used to perform imaging in the near-infrared and mid-infrared spectral regions. Note that the mid-infrared spectral region is not generally accessible via Hadamard encoding masks based on LC technology since any LC material generally has strong absorption bands in the mid-infrared spectral region. A stationary Hadamard encoding mask of available for the visible and near-infrared spectral regions is the digital micro-mirror device (DMD), a device based on micro-optoelectromechanical systems (MOEMS) technology and developed by Texas Instruments for projector display applications. One DMD format incorporates 508,800 micro-mirrors in a 848 column by 600 row array that is 14.4 mm wide by 10.2 mm high. Each individual micro-mirror is 16 microns square and adjacent micro-mirrors are separated by a 1 μm gap. The micro-mirrors are individually addressable and rotatable by +10 or −10 degrees about the diagonal axis to produce binary “on” and “off” states. The on state has Tt determined by the mirror reflectivity and approaches 1 while the off state approaches To=0. However, the ideal condition of on and off is not realized due to diffraction of the light off of the small and periodic features of the micro-mirror device.
The DMD is an array of spatial resolution elements that can be selected as groups of super-resolution elements or as individual resolution elements consisting of a single micro-mirror. The DMD resolution elements are disposed as spectral resolution elements in the spectrometer with the columns attributed to the frequency or wavelength dimension and the rows attributed to the slit height dimension. The DMD resolution elements are utilized as spatio-spectral resolution elements in the imaging spectrograph with the columns as the frequency or wavelength dimension and the rows as a vertical spatial dimension with the horizontal spatial dimension being accessed, if desired, by translating the sample relative to the imaging spectrograph. The DMD resolution elements are spatial resolution elements in the imager with the columns for the horizontal dimension and the rows for the vertical dimension and the frequency or wavelength dimension provided by other instrumentation. If a photo-acoustic detection system to is present then the depth dimension of the sample can also be accessed by changing the modulation frequency used in the photo-acoustic detection system.
The notable features of HTS, HTI and HTSI are: a multiplexing technique using a single-element detector; uses a Hadamard encoding mask (multi-slit array) in the focal plane; sends one more than half the resolution elements to the single-element detector in an encodement; uses a number of encodements equal to the number of resolution elements desired and the number of mask elements (pixels) in the stationary Hadamard encoding mask (a moving Hadamard encoding mask has 2N−1 mask elements); each encodement contains a different combination of one more than half the resolution elements; the primary data is the encodegram, a record of detector response versus encodement number; and uses a FHT of the encodegram to decode the encodegram and generate the spectrum or image. Additionally, HTS is a dispersive technique using a single-element detector.
However even considering these notable features of HTS, HTI and HTSI, there is still a need for a multiplexed spectral imaging system that can accomplish its multiplexing and scene scanning without macro-moving parts while maintaining the advantages of a DMD-based Hadamard transform spectral image collection system. Additionally, the present invention proceeds upon the desirability of providing such system that can be digitally controlled, enabling adaptive measurement of spectral image parameters.
Beginning with Martin Harwit's pioneering design, U.S. Pat. No. 3,720,469, extending M. Golay's fundamental work of the 1940's to Hadamard transform imaging spectroscopy, many novel spectrographs have been conceived that apply fixed optical masks mechanically scanned with respect to the object being imaged. We note two relevant recent examples that employ novel mask designs and coded apertures schemes similar to the present invention, but different in that they require mechanical translation of a fixed mask.
Stephen Mende, in U.S. Pat. No. 5,627,639, discloses various coded aperture methods for imaging spectrographs. He employs novel Hadamard mask including the possibility of a spatial light modulator (LCD) but still requires the scene being imaged to be translated with respect to the mask.
Whereas, in “HADAMARD imaging spectrometer with microslit matrix”, Rainer Riesenberg; Ulrich Dillner; Proc. SPIE Vol. 3753, p. 203-213 (October 1999), the authors describe a fixed novel MEMS mask consisting of tiny mirror pixels arranged in a coded pattern. However, the mirrors are fixed and, again, the mask is placed at the entrance of an imaging spectrograph and mechanically scanned across the aperture in order to Hadamard encode the input.
Given the recent developments in micro-mirror array technology, pioneered by Texas Instruments™, the ability to digitally control the position of the individual mirrors has enabled several novel spectrograph designs that no longer require scanning in the conventional sense. For example, in “Characterization of a digital micromirror device for use as an optical mask in imaging and spectroscopy”, Kevin J. Kearney; Zoran Ninkov; Proc. SPIE Vol. 3292, p. 81-92 (April 1998), the authors propose a design for a Multi-Object-Spectrograph (MOS). Their system involves an imaging spectrograph with a micro-mirror array placed at the entrance image plane. The mirrors are adaptively used to select a single object per line of the array but at any location within that line, thus allowing the spectrograph to query multiple objects in the scene no longer necessarily vertically aligned as required by conventional line-scanning spectrographs. However, in a sense, this system uses a spatial light modulator to avoid the multiplexing taken advantage of in the present invention that intentionally records overlapping spectra on the focal plane array.